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Abstract — P2P systems provide a scalable solution for dis- 
tributing large files in a network. The file is split into many 
chunks, and peers contact other peers to collect missing chunks 
to eventually complete the entire file. The so-called 'rare chunk' 
phenomenon, where a single chunk becomes rare and prevents 
peers from completing the file, is a threat to the stability of 
such systems. Practical systems such as BitTorrent overcome this 
issue by requiring a global search for the rare chunk, which 
necessitates a centralized mechanism. We demonstrate a new 
system based on an approximate rare-chunk rule, allowing for 
completely distributed file sharing while retaining scalability and 
stability. We assume non-altruistic peers and the seed is required 
to make only a minimal contribution. 

I, Introduction 

The marvel of peer-to-peer (P2P) networks is their scal- 
ability and robustness. Both of these attributes are due in 
turn to the distributed nature of such systems. In a P2P file 
sharing system such as BitTorrent Q), a large file is split into 
many chunks. A peer who downloads a chunk can immediately 
start uploading that chunk to other peers, contributing to the 
resource pool of the sharing network. 

To ensure the availability of all chunks of the file at all 
times, at least 1 seed (who has the complete file) is assumed 
to stay in the network at all times. However, in an open system, 
where peers are arriving according to a random arrival process, 
the presence of a single seed does not guarantee stability. It 
has been observed and demonstrated through various analytical 
models (|2), 0, flU, 0) that if the peers contact each other 
and download chunks in a purely random fashion, a single 
chunk might be driven to near extinction, causing peers to 
stay in the system for a long time and driving the number of 
peers in the system to infinity. In this scenario, the rare chunk 
is difficult to obtain, because there is a high probability that 
the randomly contacted peer does not have it, and peers who 
have the rare chunk tend to leave the system quickly since 
they probably already have every other chunk. Thus the rare 
chunk becomes progressively rarer as new peers accumulate. 

It is known that the rare chunk issue can be avoided through 
altruistic behavior of the peers [5|, [6|. In this model, peers 
who complete the file stay in the system for an additional 
random time to aid the remaining peers. It has recently been 
proven [7] that it is sufficient for peers to remain in the network 



for a time that is on average equal to the time it takes to 
download a single chunk. Unfortunately in real networks, the 
altruistic peer assumption may not hold. 

A similar stabilizing effect is observed if the upload capacity 
of the seed is large enough to maintain a balanced chunk 
distribution in the system [8|, |7|. However, in such scenarios, 
the demand on the seed scales with the number of peers in 
the system, reducing the scalability of the protocol. 

BitTorrent addresses the 'rare chunk' issue by forcing peers 
to download the rarest chunk first. This rule necessitates a 
centralized search in the network to track the rarest chunk, 
and the peers who possess that chunk. The tracker is the only 
centralized piece in the BitTorrent protocol, and presents a 
single point of failure for the system. It is desirable to replace 
this mechanism by a distributed rule that approximates the 
'rarest chunk first' rule for downloads. 

In this paper, we present a new P2P file sharing protocol 
that is provably stable. Our protocol is completely distributed, 
and fully scalable, avoiding the pitfalls of a centralized tracker 
or a privileged seed. Moreover, we assume that peers who 
complete the download leave immediately. Stability depends 
on a probabilistic local rule that peers follow to approximate 
prioritizing the rare chunk. 

We model an open system, where peers are arriving ac- 
cording to a Poisson process. In our model, the current chunk 
profiles of all the peers in the system defines the state for a 
Markov process. This is in line with the model described in 1 8 1 . 
In contrast with deterministic fluid models such as the one used 
in 1 6 1, we attempt to directly prove the stability of the dynamic 
system. This approach is more reassuring, as a well defined 
relationship between the stability of the dynamical system and 
that of the fluid models is yet to be formulated [3|. Our model 
and assumptions are detailed in sectio n [Tlj 

The protocol is presented in section [ill] It is a modification 
of the 'majority' rule that was first proposed in |9|, 1 10], where 
the authors used simulations and large system limits to argue 
that the rule leads to a stable system. We have since been able 
to formally prove stabililty for the special 2-chunk case 
(see section M for a brief discussion), however stability for the 
general case remains a conjecture. Our proposal implements 
a stricter rule to keep the rare chunks in the system longer, 



and allows us to prove stability in general for any number of 



chunks and any arrival rate. The proof, presented in section IV 



employs an unconventional Lyapunov function, which is the 
main contribution of this paper together with the new protocol. 
The form of the Lyapunov function is quite novel and might 
be useful in proving the stability of similar algorithms. 

II. Problem definition 

A file is divided into k chunks, to be distributed in a 
P2P system. Peers enter the system at Poisson rate A and 
leave immediately upon receiving all k chunks. We make the 
following assumptions: 

• There is always exactly 1 seed in the system (denoted by 
s), who has all the chunks. 

• At a Poisson rate of 1, each peer can sample randomly 
with replacement up to 3 peers from the current popula- 
tion U{s}. (A peer is allowed to sample itself.) 

• At the time of sampling, the peer can choose to download 
at most 1 chunk which it does not already have, but shows 
up in the sample. The download is assumed to happen 
instantaneously. 

• A newly arriving peer arrives with no chunks. 

Let S be the total number of peers in the system (including 
the seed). Note that each peer (including the seed) can be 
sampled, on average, at most by three other peers per unit time. 
Therefore the average upload bandwidth per peer is bounded, 
and does not scale with A or S. 

We seek a rule by which a peer can decide which chunk (if 
any) to download at each time slot from the current sample. 
This rule is assumed to be a function of the peer's current 
chunk profile, and the profiles of the sampled peers. (The rule 
could possibly depend on the past observations of the peer, 
as well as A and k. However the rule we propose will not 
depend on these.) We require that the chosen rule stabilizes 
the stochastic system in the Lyapunov sense. 

III. Solution: Common chunk protocol 

Let Si —# peers who have chunk i including the seed, i E 
{l,...,k}. 

So = # peers who have no chunks. 

&i = S — Oj. 

T-i =# peers who have only chunk i missing. 

Definition 3.1: A chunk in a sample of 3 peers is rare if 
exactly 1 peer in the sample has that chunk. A chunk is called 
a match if it is contained in the sample but not in the sampling 
peer's profile. 

We define the rule as follows: 

• Peers with no chunks sample 3 peers at random and 
choose to download a chunk that is a rare match. If there 
is more than 1, they pick randomly among them. If there 
are no rare matches, the peer skips this time slot without 
downloading. 

• Peers who have more than zero, but less than k— 1 chunks 
sample only 1 peer at random, and download a chunk at 
random among those that match (no rare match required). 
Skip if there is no match. 



• Peers who have k — 1 chunks sample 3 peers at random. 
Download only if every chunk that the peer has appears at 
least twice in the sample, and there is a match. Otherwise 
skip without downloading. 

Roughly, the first item is meant to stop arriving peers from 
acquiring a common chunk as their first chunk. The last item 
attempts to keep rare chunks from leaving the system. This 
should balance out the chunk distribution in the system and 
provide stability. 

Note that by sampling only 3 peers, we are requiring the 
bare minimum that allows a majority rule. By sampling more 
peers, one could clearly do better, however our main purpose 
here is to demonstrate that stability is possible even in this 
restricted setting. We discuss sampling more peers and other 
performance enhancing heuristics in section M 

In this paper, we model the proposed system by a Markov 
process with state space X described by the peers currently in 
the system and their chunk profiles. The description of the state 
space is essentially identical to that in [8|. Let g(x, x') denote 
the entries of the generator matrix of this Markov process. For 
any function on the state space, 

Definition 3.2: The drift A/(x) of a function /(x) is 
defined as 



A/(x) 



£<z(x,x')(/(x')-/(x)). 

x'^x 



We use the following well known tool for the proof: 
Theorem 3.3: [Foster- Lyapunov] Let L be a function on the 
state space with drift AL. Let L > and let {L < 1} be a finite 
set for any finite constant I > 0. If for an e > 0, AL < — e 
on the set S > c, for a suitably chosen constant c, then the 
Markov process is positive recurrent. 

IV. Proof of stability 

We will split the proof into two cases according to whether 
A < Jjr or A > Jjr. In each case, we will show the stability 
of this system by demonstrating a Lyapunov function for it. 

Let r be the total rate of downloads. dS^ is the virtual 
rate (stochastic intensity) at which a peer with no chunks 
downloads chunk i, and dS~ is the virtual rate at which a 
peer who is lacking only chunk i downloads i and leaves the 
system. We will need the following lemmas: 

Lemma 4.1: r > -^ where tq = '^2 i dS~f . 

Proof: Write S t = S + B t + % to define Sj, B % is the 
number of peers who lack chunk i and have at least 1 and at 
most k — 1 chunks. We can write 

r > S Q d(So) + Tid(Ti) + L\d(L\) 

where d(.) denotes the virtual rate of downloads for an indi- 
vidual peer in each group. By definition, d(So) — 2~2i dS^ > 
dSf for any j. Also 
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3S 2 s 
where the last inequality follows because A ' is the prob- 
ability of a rare match, but dS^ might be smaller due to 



the possibility of multiple rare matches. It is left to note Also 
d(Bj) > — g 4 -, To argue this, note that d(Bi) is the total virtual 
rate of downloads per peer for peers in Bi, and a sample of 
chunk i is sufficient to result in a download (even if chunk i 
is not chosen to be downloaded). The probability of sampling 
chunk i in 1 go is at least | the probability of sampling chunk 
i in 3 tries, which in turn at least as large as dS^~ . We can 
now write 

r > (So + \BddS+ + ^dS+ > ^dS+. 



Here we argue that x + y + z = 1 
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Lemma 4.2: 
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if 5 > 12. 



Proof: Argue as in the previous lemma that for So, a rare 
match of i is sufficient for a download. For Bi, a simple match 
is sufficient and % > -jr 1 . Therefore 



r > (35 + Bi) 
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Here we argue that x + y + z = 1 



for any i. 
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Since max,; Si > ^r- 1 



we have the result when 5 > 12. 
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The first two terms correspond to the the arrival of a new 
peer. The second term is the drift due to a peer with no chunks 
downloading chunk i. The last term corresponds to the event 
where a peer leaves the system after having downloaded chunk 
i. 

The inequality is due to the fact that we omitted terms 
corresponding to transitions which keep S and Sq constant, 
and in the last set of terms, for each individual i we ignored 
the terms corresponding to Si and So, These transactions can 
only decrease L^. 

Since dS^~ < min^ -55- and T$ < Sj, Vj 7^ i, the last 
term satisfies 
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The drift of L is 



ALi = kX-r < - 



From lemma |4~2] we know 
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, for any i. 



Picking i to be the rarest chunk, we observe that r > g + e 
whenever 5 > 3fc 3 . 

Now assume A > 
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where C is a constant to be chosen later. 

We will calculate the drift of L in two parts. A download of 
chunk i decreases S{ by one and leaves all other Sj unchanged. 
A new arrival increases each Si by one. Therefore 

Aii = kX - r. 



Since £\ ^- + ^ < (k + 1)A, we may write 

AL< 8 4 + X + (C + l)kX-Cr- ^' 1 J S 
S e • e s ° 
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Now we are ready to show 
Theorem 4.3: 

AL< -e 



with e > and C = 108efc 3 whenever 5 > 4Cfce 3CAfe e 
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Proof: Since we assume that 5*_ < 1, we are left with 5 



terms which can be written as 
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, which 



AL < 1 + A + (C + l)kX - -CfcA < -e. 



If r < 2fcA, then by lemma 
dSf > "S 1 , t 
used the bound 
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0. Again by the same reasoning as above, if So > 
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It is clear that both Lyapunov functions that are used satisfy 



A. m-sampling 

The rule for the peers that have all but one chunk can be 
eased as follows. A peer in X!; samples m peers at random 
with replacement instead of 3. Allow a download only if each 
chunk other than i is observed at least twice in the sample of 
m peers. This would ensure none of the chunks which leave 
are rare. Sampling more peers increases the complexity of the 
system (decreases locality), but allows for a more efficient 
search (peers could leave earlier). One could pick m to strike 
a good trade-off between complexity and performance. 

The proof of stability generalizes to this case with little 



modification. In 



0. 



note that dS A < min, 



-. There- 



fore the last term would be bounded by (™ l ) ^-, which can be 
bou nded by 1, provided we modify the bound on S in theorem 



4.3 



with 



The rest of the proof goes through unaltered. 



B. Rare chunk rule 

The original rule proposed in O, ifTUl was as follows: All 
peers sample 3 other peers with replacement, and download 
only if there is a rare match. As noted before, this is a 
minimalist approach to approximation by a majority rule. The 
difficulty that arises in trying to prove the stability of this 
system is that the majority rule does not in general favor the 
rare chunk, but rather inhibits the common chunk. These two 
goals turn out to be identical in the special 2-chunk case, for 
which it has been possible to find a Lyapunov function: 

Theorem 5.1: 

L = 2(25 + Si + S a ) + (Si - S 2 ) 2 , S > 30A(20A + l) 2 

is a valid Lyapunov function for the 2-chunk system with the 
rare chunk rule described above. 

In the interest of keeping our focus, we omit the proof of 
this result. We will only remark that the first term will be 
decreasing whenever there is sufficient balance in the system, 
and the second term turns out to be always decreasing due to 
the rare chunk rule, and makes up for the increase in the first 
term when the system is in severe imbalance. 



the properties of theorem 3.3 We conclude that the proposed q Simulations 



system is positive recurrent for any value of A. 
V. Performance 

From a performance point of view, some aspects of the 
protocol may strike the reader as inefficient. In particular, the 
rule for leaving the system is quite strict, and may cause 
substantial delay for the peers that have all but one chunk. 
Consider a state where most of the peers have a few or no 
chunks. A sample of 3 peers needs to contain at least 2fc — 1 
chunks (1 for the missing chunk, 2 each for the rest) for a 
peer to be able to leave the system. Therefore a peer with 
fc — 1 chunks will need to wait in the system, until the system 
becomes more saturated. 

On the other hand, this ensures availability of all chunks to 
other peers, and reduces starvation in the network. This rule 
can be interpreted as forcing a degree of altruistic behavior 
and has a similar effect in terms of stability. 



We compare our proposed algorithm with the parameter m 
taking the values {3, 5, 10}, m = 3 being the original protocol 
proposed in section llllj with the rare chunk rule. 

Figure [T] shows a system with 20 chunks and A = 10. At 
time 0, only the seed is present. We can see all four systems 
reaching a stable state. The total number of peers for m = 5 
and m = 10 behave roughly similar to the simple rare chunk 
algorithm, where m = 3 hovers slightly above the others due 
to the stricter rule keeping peers in the system for a longer 
time. The same behaviour is observed in figure [2] where all 
systems relax in a similar manner from an initial population 
of 1000 peers, all of which lack the same chunk. 

As reported in Q, (9), the rare chunk rule seems to 
provide stability despite the lack of a conclusive proof in this 
direction. On the other hand, the newly proposed protocol 
performs competitively (and even more so with a suitably 




— Rare chunk 

— Common chunk, m=3 

— Common chunk, m=5 

— Common chunk, m=10 



Fig. 1. Reaching steady state from an empty system, k = 20. A = 10. 




Fig. 2. Relaxing from severe imbalance. 1000 peers all lack a single chunk 
at initialization, k = 20. A = 10. 



chosen parameter to) while having the advantage of a formal 
stability guarantee. 

One should note that the total queue size is not necessarily 
the best indicator of performance. It would be interesting to 
see whether there are differences in the mean sojourn time of 
the peers with different protocols, and how such metrics might 
be affected by the parameter to. These topics will be subjects 
of further research. 



seed can stabilize such systems, these properties are usually a 
luxury in real world networks. 

In this paper we have demonstrated that a completely 
decentralized, stable peer-to-peer network is possible, even 
with completely non-altruistic peers and a single seed with 
minimal upload capacity. While earlier work has hinted at this 
result with heuristics and simulations, it had proved difficult 
to come up with a provably stable scheme. Although our 
original algorithm has drawbacks in terms of performance, we 
have suggested an improvement that allows trading locality 
for performance. Our proof was easily adapted to this case, 
which suggests that the methods presented here might allow 
for stability guarantees for other algorithms. 
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VI. Conclusions 

Peer-to-peer schemes such as BitTorrent have been remark- 
ably successful in revolutionizing the way files are spread in a 
network. Still, it is desirable to completely decentralize such 
protocols to avoid the pitfall of a single tracker. Naive attempts 
at such schemes have been plagued with the 'rare chunk' 
syndrome, which causes instability. While it has recently been 
shown that relatively minor altruistic behavior or a powerful 



